Skip to content

[Backend Tester] Report quantization and lowering times #12838

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 82 commits into from
Aug 12, 2025
Merged

[Backend Tester] Report quantization and lowering times #12838

merged 82 commits into from
Aug 12, 2025

Conversation

GregoryComer
Copy link
Member

@GregoryComer GregoryComer commented Jul 24, 2025 •
edited
Loading

Track and report the time taken to quantize and lower in the backend test flow. Include this information in the generated report for each test case.

Example output (from testing add operator):

Test ID Test Case Backend Flow Result Quantize Time (s) Lowering Time (s)
test_add_dtype_float32_coreml test_add_dtype coreml coreml Success (Delegated)   0.69
test_add_dtype_float32_coreml_static_int8 test_add_dtype coreml coreml_static_int8 Success (Delegated) 8.73 0.88

@GregoryComer
Update
f120e70 
[ghstack-poisoned]
@GregoryComer
Update
0fb85e6 
[ghstack-poisoned]
@GregoryComer
Update
4d8d844 
[ghstack-poisoned]
@GregoryComer
Update
dc12b40 
[ghstack-poisoned]
@GregoryComer
Update
ead0616 
[ghstack-poisoned]
@GregoryComer
Update
0f13676 
[ghstack-poisoned]
@GregoryComer
Update
b0b01f2 
[ghstack-poisoned]
@GregoryComer
Update
8b9c9ef 
[ghstack-poisoned]
@GregoryComer
Update
06bf03a 
[ghstack-poisoned]
@GregoryComer
Update
2f8f49b 
[ghstack-poisoned]
@GregoryComer
Update
8ca7766 
[ghstack-poisoned]
@GregoryComer
Update
bffb95f 
[ghstack-poisoned]
@GregoryComer
Update
d21492b 
[ghstack-poisoned]
@GregoryComer
Update
e2c4ea5 
[ghstack-poisoned]
@GregoryComer
Update
8230848 
[ghstack-poisoned]
@GregoryComer
Update
2a1f564 
[ghstack-poisoned]
@GregoryComer
Update
b35e7b1 
[ghstack-poisoned]
@GregoryComer
Update
5c4c6ce 
[ghstack-poisoned]
@GregoryComer
Update
9397803 
[ghstack-poisoned]
@GregoryComer
Update
9dfeb5a 
[ghstack-poisoned]
@GregoryComer
Update
ff5c4a5 
[ghstack-poisoned]
@GregoryComer
Update
42a5de5 
[ghstack-poisoned]
@GregoryComer
Update
402d8f5 
[ghstack-poisoned]
@GregoryComer
Update
34d3ab3 
[ghstack-poisoned]
@GregoryComer
Update
1105e04 
[ghstack-poisoned]
@GregoryComer
Update
482bd21 
[ghstack-poisoned]
@GregoryComer
Update
ea548b7 
[ghstack-poisoned]
@GregoryComer
Update
4108f54 
[ghstack-poisoned]
@GregoryComer
Update
7ef236b 
[ghstack-poisoned]
@GregoryComer
Update
4a58c9d 
[ghstack-poisoned]
@GregoryComer GregoryComer mentioned this pull request Jul 24, 2025
Merged
@GregoryComer GregoryComer requested a review from digantdesai July 24, 2025 23:56
@GregoryComer GregoryComer mentioned this pull request Jul 25, 2025
Merged
digantdesai
digantdesai reviewed Jul 28, 2025
View reviewed changes
Copy link
Contributor

@digantdesai digantdesai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

GregoryComer added a commit to GregoryComer/executorch that referenced this pull request Jul 31, 2025
@GregoryComer
[Backend Tester] Report quantization and lowering times
1809933 
ghstack-source-id: 22aa30d
ghstack-comment-id: 3115355420
Pull-Request: pytorch#12838
@GregoryComer
Update
dd09555 
[ghstack-poisoned]
@GregoryComer
Update
f1db3a0 
[ghstack-poisoned]
@GregoryComer
Update
e0700b2 
[ghstack-poisoned]
@GregoryComer
Update
f260b50 
[ghstack-poisoned]
This was referenced Aug 8, 2025
Merged
Merged
Merged
@GregoryComer
Update
f261355 
[ghstack-poisoned]
@GregoryComer
Update
c3a24f9 
[ghstack-poisoned]
@GregoryComer
Update
1697cbc 
[ghstack-poisoned]
@GregoryComer
Update
b94b45e 
[ghstack-poisoned]
This was referenced Aug 12, 2025
Merged
Merged
Closed
Merged
Merged
Merged
@GregoryComer
Update
7e1a002 
[ghstack-poisoned]
@GregoryComer
Update
a628d29 
[ghstack-poisoned]
Base automatically changed from gh/GregoryComer/91/head to main August 12, 2025 19:02
@GregoryComer
Update
1d34f49 
[ghstack-poisoned]
@GregoryComer GregoryComer added the release notes: none Do not include this in the release notes label Aug 12, 2025
@GregoryComer GregoryComer merged commit 5563051 into main Aug 12, 2025
101 of 102 checks passed
@GregoryComer GregoryComer deleted the gh/GregoryComer/92/head branch August 12, 2025 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@digantdesai digantdesai digantdesai approved these changes

@cccclai cccclai Awaiting requested review from cccclai cccclai is a code owner

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. release notes: none Do not include this in the release notes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@GregoryComer @digantdesai @facebook-github-bot